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1 . INTRODUCTION 



The discrimination problem may in a sense be considered 
that of multiple classification, i.e., an individual Z is 
known to belong to just one of j categories or populations 
and it must be classified into one of these populations on 
the basis of what is known about Z and the existing populations. 
The problem lends itself to a statistical approach when avail- 
able information about Z is in the form of observed values of 
random variables which have probability distributions for each 
of the different categories or populations* 

Discriminatory analysis in its historical evolvement is 
documented with an extensive bibliography by J* L* Hodges in 
[ 1 ]. 

The history of discriminatory analysis may be represented 
in several broad phases of development. A Pearsonian stage is 
Identified with the introduction and use of his coefficient of 
racial likeness* This stage is considered to be followed by 
a Fisherian stage associated with the introduction of the linear 
discriminant function and this stage is followed by a Neymanr 
Pearson stage and a contemporary Waldian stage. The latter two 
reflect the introduction of the concepts of probability of 
misclasslf Ication and that of risk into the realm of discrim- 
inatory analysis. 

For simplicity and ease of computation the discrimination 
problem will be considered only in the two population case, i.e*, 
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the Individual Z is known to be distributed over some space 
according to distribution F or according to distribution G and 
it is desired to decide which of the distributions Z has on the 
basis of the observed value z. 

An approach to this classification problem is one that is 
dependent on the amount of information assumed to be known about 
F and G* This approach allows the problem to be segmented into 
three types; 

(1) F and G are completely known — On the basis of an 
observation of Z, the problem is to determine which is the dis- 
tribution of Z. Treatment of this problem has been extensive 

V 

and its solution lies within the Neyman-Fearson lemma • 

(2) F and G are known but complete knowledge is lacking 
in its parameters — F and G are of the same family of dis- 
tributions but differ parametrically and on the basis of an 
observation on Z, the problem is to determine which is the dis- 
tribution of Z. Hodges and Fix discuss this in [z] and identify 
the most familiar example of this process as the linear discrim- 
inant function where the assumption is made that F and G are 
p-variate normal distributions having the same (unknown) co- 
variance matrix* It is noted that the approach is reasonable 

if the assumptions are well founded but validity is questionable 
if the populations are obviously not normal or if they are normal 
but with obviously unequal covariance matrices. 

(3) F and G are completely unknown -- Nothing is assumed 
about F and G other than their existence and on the basis of an 
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observation on Z, the problem is to decide which of F and G is 
the distribution of Z* 

The last problem type is a problem of nonparametric class- 
ification and is the area of concern of this thesis. The area 
of nonparametric classification has its possibly first published 
treatment in Hodges and Fix's [2] and [3]. In these papers 
Hodges and Fix considered the two population problem, however, 
they noted that the approach general, has optimum properties 
for large samples and applies to cases where there are more 
than two populations to be discriminated. In [3] a comparison 
is made of the nonparametric approach and the linear discrim- 
inant approach, assuming both populations to be normal with 
equal covariance matrices. 

Eaton in [4] and Hager in [s] extend the work of Hodges 
and Fix to one dimensional exponential populations. This paper 
continues the investigation of exponential populations when the 
distributions are bivariate exponential. 

Section 2 will introduce and summarize the concepts and 
methodology of [3]. Section 3 will apply these concepts to 
calculate a probability of misclassificatlon utilizing two 
different distance functions to the two population problem when 
both populations are bivariate exponential. Section 4 will 
present the conclusions and recommendations evolving from the 
effort set forth in Section 3. 
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2. PERFORMANCE OF A NONPARAMBTRIC DISCRIMINATOR WHEN THE TWO 
POPUIATIONS HAVE NORMAL DISTRIBUTIONS WITH 
EQUAL COVARIANCE MATRICES 

The two population classification problem will be defined 

in the following manner. Let X , X , •**, X be a sample from 

1 2 n 

a p'variate distribution F and let Y. » Y., ***> Y be a sample 
from the p-variate distribution G. An observation z is made 
and it is known to be distributed either as F or as G. The 
problem being to assign z to one of these two. 

The approach utilized for the nonparametric procedure will 
be through the concept of nearness. A distance function is 
defined in the p-dimensional sample space which allows a ranking 
of the combined samples according to their nearness to z. With- 
in this framework, z would be assigned to the F population if 
most of the nearby observations are X's and assigned to G if 
most of the nearby observations are Y's. For simplification, 
the sample sizes are assumed to be equal, i.e., m*m. As an 
assignment criteria, an odd integer k is selected and z is 
assigned to that distribution from which came the majority of 
the k nearest observations. The case studied in this thesis 
will be when k ® 1, the rule of the nearest neighbor. 

In C 2 ] it is shown that several of these nonparametric 
discriminators have asymptotically optimum performance as m 
and n tend to infinity. By this is meant that probabilities 
of misclasslf ication. 
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P (Z is assigned to G | Z came from F) 
P (Z is assigned to F | Z came from G) 



will approach the theoretical minimum values obtainable if F 
and G were completely known as the sample size (m^n) tends 
to infinity# 

In Cs] Hodges and Fix investigate the probabilities of 
misclassif ication of the nonparametric procedure when the 
sample size is small and compare the results with the linear 
discriminant function probabilities of misclassif ication# The 
populations are assumed to be normal with equal covariance so 
the linear discriminant function is optimal and the nonpara- 
metric procedure can be compared against the optimum for a 
comparison as to how much discriminating power is lost when 
the sample sizes are small# 

This Section will present some of the concepts and results 
of [3] developed in establishing this comparison# 

Initially^ it is stated that the problem can be reduced 
by considering linear tranrfornia cions on the observation space 
so that F and G will always have the identity covariance matrix, 
i#e#, the p transformed measurements are independent in each 
population and that each measurement has unit variance# Also 
the expectation vector of F can be placed at the origin and 
that of G on the positive first axis# Thus only two parameters 
p and X are required to specify the transformed populations 
where 
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X » E (first coordinate of Y) 

* distance between the means of the transformed 
populations* 

For the linear discriminant function, P, and are 

1 2 

unchanged by this transformation, hence there is no loss of 
generality. P^ and P^ for the nonparametric discriminators 
are also unaffected since such linear transformations map the 
totality of possible distance functions of the original space 
one to one into the totality of the new space* 

The univariate normal case is considered in depth due to 
its computational simplicity both for the linear discriminant 
function and the nonparametric procedure* 

Considering the linear discriminant function first, the 
univariate case eliminates matrix computation and allows the 
classification problem to be stated as; compute the arithmetic 

X+Y 

mean of the sample means, , and assign Z to the population 

2 

whose sample mean lies on the side of (X + Y)/2 as does z 
itself. An error is committed then if and only if; 

Z > (X + Y)/2 and Y > X 
or 

Z < (X + Y)/2 and Y < X. 

Hence , 

P^= P(Z > (X+Y)/2, Y > X) + P(Z < (X+Y)/2, Y < X) 

and Pj^ due to the syroetry of this particular problem. 

In [3] it is shown by defining new variables of X, Y and Z, the 
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limiting value of as is .5. 

Considering now the nonparametric procedure, again n=m 
and the populations are univariate nonnal. The discriminator 
will be the case of k * 1, i#e., assign Z to the nearest sample 
(rule of the nearest neighbor)* The distance function utilized 
to measure the degree of nearness will be; 

A (x,z) » max [x. - z | 
i=i ^ i 

This function, A (x,z), describes a hyper-cube in p space and 
is only one of many functions that could be used* In the case 
of p => 1, A (x,z) corresponds to Euclidean distance. 

To arrive at the error probabilities a conditional prob- 
ability F^(z) is introduced and defined as the probability that 
the nearest of the 2n sample observations to Z is a Y given that 
Z “ z. Then 

00 

(1) P = E [P fz)] - f fz<*> 

1 Z 1 J ^ 1 

where the distribution of Z, 

f^(z) - iU/Tiry y» 

is the distribution of the x's, F. 

i 

In order to calculate P.(z) in general the quantities H_( 6 ) 

1 ^ 

and K (6 ) are defined as follows: 

Z 

11 (6 ) = P (the distance of X from z < 6 

z 

H (6 ) = P (the distance of Y from z < 6 ). 

C» 
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The explicit forms of H (^; ) and K (^, ) and therefore P , will 

Z Z V 1 

be dependent on the distance function utilized to express the 
nearness* 

In [ 3 ] the distance function utilized is 
P ^ 

A (x,z) = max |x', - 2. | 
i=l 

and since the univariate case is considered, H (6 ) and K (6 ) 

Cg Lg i 

can be expressed as: 

) = P (|x-zl < 6 ) 

K (5' ) “ P (1y-z1 < 6 ) 

Cg 

K 

The formulation of P (z) in terms of H (6 ) and K (6 ) can be 

1 Z V Z ^ 

done by considering the distance from z to each of the Y obser- 
vations, 

IYj^-zI , 1y^-z1 , ••% |y^-z1 . 

Hence with the assumption of independence and of equiprobable 
events , 

P(max 1 y^-z1 < ^ ) = P(1Yj^-z1 <6) 




Therefore, 

t ft 

P(min 1 y -z| >8') = P(1 y -z| > 8?;) **• 
1 1 

P(|y -zl > 6 ) = (1-K (8 ))" 
ri ^ 

or 

P(min 1 y^-z 1 < P “ 1 -(l-K^^ ))". 
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The density of the minimijim distance between Y and z is then 



n ))"" dK (6 S . 

^ z 

Similarly the minimum distance between and z can be treated 

with the result that 

I I " 

P(min |X -z > 6 ) “ (1-H (6 )) , 
i Z 

Now, P^(z) = P (nearest observation to Z is a Y given that 



00 ^ 

Z»Z) « J P(rain Ix «zl > 6 1 min 1 y.“z1 = j ^^mink 

iB 

In terms of H„(6 ) and K„(6 ) then, 

2 ' Z 

p p n n-1 

(2) P (z) = n (1-H (6)) (1-K (& )) dK (6) , 

1 J 4 ^ X ^ i 

O 

Equations 1 and 2 form the basis for computations for the 

tule of nearest neighbor for any p and any distance function, 

the explicit form of H™('6 ) and K (6 ) being dependent on the 

^ Z ‘ 

distance function. 

Figures 1 and 2 illustrate the conq>arison of the linear 
discriminant function and the nonparametric discriminator error 
probabilities when the distance function is A, k = 1 and the 
populations are univariate normal. 

It is shown in [3] that for large n and in general. 




aC?) 1- 

_ f (z)+g(z) J 



r g(z) f(z) dz 
, 00 f(z>+g(z) 



> 



the value which by Schwartz's inequality cannot exceed % . 
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nonpararaetric 
, discriiainator 




FIGURE 1 

Probability of Error vs. n for Linear Discriminant 
Function and Nonparametric Discriminator, k = 1. 

Both populations are univariate normal. X = distance 
betv;een the means. Distance function is a . 
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Probability of Error P^ vs, X for Linear Discriminant 
Function and Nonparametric Discriminator, k = 1, 

Both populations are univariate normal. Dotted line 
indicates nonparametric procedure, n = 1 is identical 
for both. Distance function is A , 
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The depexxdence o£ on the distance function was mentioned 
earlier and Hodges and Fix In [ 3 ] present some results showing 
how Is affected by alternative distance functions. 

In considering these other distance functions, the sample 
populations are assumed to be bivariate normal allowing a greater 
choice of distance functions than can be encountered when p=l. 

The function A (x,^ls In this case, 

A ((x^,x^), “max ^ ’ 

a locus of points centered at z In the form of a square. 

Euclidean distance, describing a circle centered at z Is 
defined as, 

A ((x ,x), (z ,z )) - ((X -Z + (X -Z . 

112 12 ll 22 

A distance function describing a rectangle centered at z 

* ;r 

In the ratio of one to three and vhoWe sides are parallel to 
the axes Is, 

l.e., a rectangle whose vertical dimension Is three times Its 
horizontal dimension and the common multiple being the 
max (IXj^-Zj^l , )‘ 

Similarly a distance function describing a rectangle 
centered at z In the ratio of three to one and having sides 
parallel to the axes Is, 

A3 ((x^,X2>, (z^,Z2» “max ( 3 |x^-Z^| , Ix^-Z^l ) • 



20 



Limited computed results are offered in [3] regarding the 
use of the^'^ distance functions A, A^ and A^ , The com- 
parative results are illustrated in Figure 3 in the form of a 

f 

^lot P (0,0) and n. P (0,0) is the conditional probability of 

error given that z is at the origin. It was presented because 

it was remarkably consistent with the value of P^. 

Comparison of the results in Figure 3 concludes that in 

this case there is little difference in P whether A or A is 

1 1 

used. However there is great effect with respect to the use 
of the other distance functions and hence a burden is placed 
upon the statistician for selecting the appropriate distance 
function. 

Though not covered in this summary, Hodges and Fix in [33 
investigate to a limited extent: 

(1) The nonparametric discriminator using A as a distance 
function with k » 3 for the univariate and bivariate normal 
distributions. 

(2) The nonparametric discriminator using A as a distance 
function with k ■ 1, n =* 1, and p ^ 2. 
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?l(0,0) 





FIGURE 3 

Probability of Error P^(0,0) vs, n for Various Distanc 

Functions for the Nonparametric Procedure, h = 1, 

Both populations are bivariate normal, 1 =2. 



22 



3* NONPARAMETRIC CLASSIFICATION ERROR PROBABILITIES FOR TWO 
DIMENSIONAL EXPONENTIAL POPULATIONS 7GR 
TWO DISTANCE FUNCTIONS 

Hodges and Fix's work in [ 2 ] and [3] of comparing the 
performance of the linear discriminant function and that of the 
nonparametric procedure is applied to the two population expo- 
nentially distributed case by Eaton in [4] and Hager in [S]# 
Eaton studies the small sample performance and Hager extends 
Eaton's work to larger sample size and also gets some asymp- 
totic results* In both [ 4 ] and [S] the study is limited to one 
dimension, i.e*, p * 1. Hodges and Fix's major effort was for 
the case p « 1, with only limited presentation of results for , 
p ^ 2 for the nonparametric procedure and none for the linear 
discriminant function. 

This section will consider the following: 

(1) The two population classification problem when the 
distributions are two dimensional exponential. 

(2) Formulation of the error probability P^^ ® P(Z is 
assigned to g|z came from F) , utilizing the nonparametric pro- 
cedure when the distance function is: 

(a) Hyper-cube, 

p 

A (x,^ ** max [ | x -z . | ] 
i-1 i ^ 

(b) Hyper- sphere, 

2 2 h 

A.(x,z,) - [(x -z ) + •••+ (x -z ) ] 

1 11 P P 
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The density functions of F and G will be denoted by 




(x ,x ) and f 
1 2 ^ 1^2 



(y 




respectively where 

X + X X ) 

'x X ° 

12 



and 



» 0 



otherwise 



-(M- y. + M- y ) 

f „ (y,,y ) “ e ^ ^ 2 2 for y s 0 

^12 ^ 2 1 2 i 1 



otherwise. 



Independence is assumed between and and also for and 



Y . 

2 



Following the procedures of Section 2, P ■ E [P (z)] . 

1 Z 1 



00 00 

1 o'* o 1 hh ^ 2 1 



dz 



> - J J P (z) f dzo Pi(z) ^7 7 (z ,z )dz 

^ z ^ z. ^ 12 12 2 1 2 ^ ^ 1 1 2 



*1 *2 



2 1 



where 



-(X z, + X z ) for z.,X ^ 0 
f_ ^ (z,,z ) = X,X^ e ^ ^ 22 1 i 



■ZiZ V 1 2 



0 



otherwise 



Pl(z) = n I [1-K2(6)] dK (6) 

6 < z < z 
1 2 

+ "J [1-H2(6)]“[1-K^(6)]“‘^ dK (6) 

z,<6 <z ^ 

1 2 



J 



n n-1 

[1-H,(6)] [1-K„(6)] dK,(6) 



z<z< 6 
1 • 2 



when z s g . 
2 1 



When z s the subscripts interchange. 



The explicit form of H (6) and K (6) will be dependent 

Z Z 

on the distance function utilized and the relationship of 6 
and z. 

The hyper-cube distance function will be considered first, 

2 2 
A (x,z)* max [|x.-z.|] , A (y.z) = max C1y.-z,|] . 

i-1 ^ ^ i=l ^ ^ 

For Z=z, and 6 s 0, H (6) and K (6) can be defined. 

Z z 



H (6) = P( max Clx.-z. 11 < 6 ) » P(max[ jx -z ], Ix^-z.]] < 6 ) 

Z i=l ^ ^ 1 . 2 2 

= P(lx^-z^l, IX 2 -Z 2 I < 6 ) = P(|x^-z^l < 6 < 6), 

assuming independence of the differences* Hence H (6) will be 

Lt 

evaluated in six regions: 
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6 



z + 6 
2 



Hj(6) - f ‘‘■‘l I %<*2> '*’‘2 * ‘ 2 ^ 'l 



z - 6 
1 



z + 6 
1 



"2- ^ 



Z.+ 6 
2 



J "■‘i I ‘x,<'‘2> 



z s 6 s z 
2 2 1 



"r ^ 



z + 6 
1 



y z ^ z 
1 2 



z + 6 
2 

I- 



J tj; (x^X; ■i=t J £x (*2) ^’‘2 *2^ '1^ ^ 

o ^ 



z^+ 6 



z + 6 
2 



J £jj^(*i) <‘*1 I ‘xj^V ‘**2 



6 s z,s z„ 
1 2 



*r ^ 



2j^+ 6 



z - 6 
1 



z + 6 
2 



J ‘x/*i> -“i I 'x '*’‘2 ‘ ^ ^ 



Zi+ 6 



z - 6 
2 



z + 6 
2 



I \ 



fv 



J> £jj <«2) <1X2 x^x Xj"" ® 



z <: z 
2 1 



iS 

j; 



Similarly 

K^(6) - P(|y^-Zj^1 < 6 ) PCIy^-z^I < 6 ) 



and will be defined as was H (6) in the six regions. Differ- 

2 

entiating K (6) yields dK (6) in the six regions. 

Z Z 
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Evaluating H ( 6 ), K ( 6 ) and SK ( 6 ) and combining terms 

z z ^ z 



yields, when z. ^ z , 
1 2 



r ii -X Z--X z n 

[l -4 e ^ sinh ^.^6 sinh 



n A .n-l 

® sinh 6 sinh 6 ] 



”^1^1 ”^2*2 

(4 e ) [p-, sinh Pi 6 cosh pi, 6 

1 2 1 



+ Pi^sinh pi 6 cosh p, 6 ] d 6 
■^2 2 



P r “^1*1 ^ 

+ n J [ 1"2 e sinh 6 + 2 e sinh \, 6 ] 



n* 



.•n 



-pi 2 -p.-2--Pi-6 



W-.I 



"M'lZn 

[ 1-2 e sinh pij ^6 + 2 e ■*■ sinh pij^S] 



r ■'^1^1 

L 2 p.j^e cosh pij ^6 - 2 p.j^e 



-pij^z^ - p.2Z2">*2^ 



cosh pij ^6 + sinh* pij ^6 ] d 6 






+ n J Ce 



+ e 



- e 






+ e 



[pi 



-|i 2^-(1^6 -H»z -h 6 -Pi 2 -Ii 2 -Pi^6-n 6 

^ ^ ^ + Pi^® ■<'"1+ ^*2^® 112 2^ 2 ;,^5 
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and when z ^ z 
2 1 

r l "^•iZi “X z„ n 

[l-4 e ^ sinh X^6 sinh X^6] 



z n-1 

[1-4 e ^ ^ sinh p. 6 sinh m-,6] 

2 1 



-^l^z^-^x z 

L4 e siiii M> 6 cos M>,6 + |x siiii 

1 2 1 2 ^ 



codi 



-X, 



•X-Zt “X-z.-X. 6 



“ J [1-2 e sinh 1 6 + 2e ‘ sinh X 6]° 



r , , ■'*=‘'2 • n s . , ->* in - l * 2»2 -'‘ l ^ , ^ ^ "-1 

[1-2 e sinh p. 6 + 2e sinh p. 6J 

2 2 



-M-^z, -|X zj^-ix z -p.,6 

[2p.,e cosh n„6 - 2|i.e 2 2 



cosh 



-^4lZl-u z -^il6 

p, 6 + 2p. e ^ ^ sinh p. 6] d6 



00 



+ n JCe 

Z 



’X 1 Z 1 -X 16 -X 2 -x^fi,;^ -^l2l“^2^2‘^1^.^2\n 
e e J 



^-ti^Zi-p,^Z2-p.j^6-n^6^n-l 






+ p-^e - (p-j^-Hi^) e 



]d6 
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p" -^iZl '^ 0^0 

Then “ J J Pj^(z) ^ dz dz^+ 



X. z *2 

P ^ 2 2 P 2 



I X e 

J 2 



P ^ "^1^1 

J P (z)^,e dz ' 

J 1 1 1 



dz 

2 



The Euclidean distance or hyper-sphere distance function 
for p « 2 is: 



A^(x,z) = [(Xj^-Zj^) + (X2-Z2>^]^ . 

Evaluation of P , P (z) , H (6), K (6) and dK (6) will follow 

1 I z z z 

the same logic as that followed for the hyper-cube. The explicit 
form will be more awkward however. 

Consider H^(5) “ P([(Xj^-zp^ + (X^-z^)^]^ < 6 ). For ease 

of notation define: 



Then, 



where 



S = X - z and T = X - z 
1 1 2 2 



H^(6) = P(S^ + T^ ^ 6^ “ I J 



-Xj^(s+z ) 

f (s) = f„ (s+z ) = X e for s + z, ^ 0 

S 1 ^ 1 



= 0 otherwise 



-X (t+z ) for t + z 20 



= 0 



otherwise. 
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The definition of S and T translates the axes to center at 



(z >z ) and H (6) is the evaluation of the circle centered at 

12 z 

(z ,z ) with 6^0. Here as before, evaluation of H (6) will 
12 Z 

be over a region which is the union of six mutually exclusive 
sub-regions. Therefore, 



6 

H (6) = f ds dt 



o o 



6 

J I 

•6 -z« 



fg|.(s,t) ds dt 




6 s z s z 
2 1 



z^6Sz, >z,Sz 



z S z S 6 
2 1 



1 2 



6 

4 J J f (s,t) ds dt 



o o 



ST 



6 



J' J ds dt 



6 J^-s^ 



-z -z 
1 2 



6 ^ Zj^s z 



■- t; 



Vi' ‘=2 



z s z ^ 6 
1 2 
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K (6) is similarly treated by defining, 

z 

V=Y^-Z2 . 

“ P(U^ + ^ 6 ^ ) » J J 

uVv^ < 6 ^ 



where 



-lJ,j^(u+z J 

f (u) = (u+z,) = li, €* for u + z ^0 

U ^ i 



and 



otherwise 



-p. (v+z ) 

f (v) “ f (vfz ) * |J. e ^ 2 for v + z ^0 

V Y2 22 2 



othen;isd. 



The functional form of will be the same as 

and dR (6) will be defined over the same regions as H (6). 

Z Z 

Evaluating H^(6), combining terms yields, 

when z ^ z , 

I 2 

2 [1 + 4e V2 . 1 1 2 2 1 1 1 2 2 



I 



-X, S“X^ v6^-s^ 



[1 + 4e ^ ^ 2 2 _4e ^ ^ 22 1 



- 4e ^ ^ 2 2 J lA^e 



-|i u-u n_i 

^ 2 



-|ji z -M.-Z -p. 6 -p. z -u z 

[-8,11 e ^ ^ 2 2 1 + 4^^ e 1 1 2 2 
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du ] d6 



P^(z) 



li 6 

I 






a 2 2 
6 -u 



/ ^2 T 



r“l "^iZl Z„ r •M" '*» 

■^nj [1 - 2 e ^ ^ sinh \ 6 + e^^22j„l 2 

-6 



X_z.-X_z. -X, s-X.^6^-s^ n 



ds] 



c 12^ 

-IJ. z, -lA.z.-ii z ;. -|i,u-m.^\/6 -u n-1 

[l - 2 e sinh |j.j^6 + e 22^^ 2 

-6 



•JA z^ -M',Zi-M.,z "l^iZi-IA-Z 

[2pi^ e cosh - 2 e cosh iAj^ 6 + |j,^6 e 2 2 

^6 [ 5^-u^ 

J >1 6^. u2 






du] d6 



+n 






-z 



■S n 

ds] 






J 



u n-1 

du] 



1 2 



. /a2 2 

6 -|i u-M.- ^ 6 -u 

r e ^ 



•li 



du] d6 
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and when z ^ z , 
2 1 



»^(z) = nj l[i + 4e 



. 4e''l^'V2 
-Xj^s-X Jd^-! ^ 



"S n 



X^e 



ds] 






[1 + 4 



6 [ 2 2 
r -(A u-ji V6 -u n-l 

j ^ 2 du] 



-Ji,z -|A,Z -Ji 6 -Ji z -Ji z 

[-8n^e ^ ^ 2 2 1 ^ 1 1 2 2 



J !^2__ du] d6 



+ n 




2 Xj^e 



-h'l- 



X z 
2 2 



z 

1 




sinh 



\/?^ 



2 



z 

1 



ds]*" 



[1 - 2^ e 



■^"fV2 




du] 



n-l 



C2H^6 



•‘"i*rV2 




du] d6 
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p (*) 

1 



-X..Z -X 6 -X.z -X z 
+ nlre 11 1 + g ll 22 



60 

- J[, 



-X s-X 



1 2 



ds] 



”Z, 



-ji.z -m.,6 -h z.-\lz r 



11"^! 



+ e 



1-1-2-2 J 



^ nT^-u^ n-1 



du] 






CM-j^e - e 



e rr2 2 

6 J 6 -u 

I ‘ ^ 



•Z, 






du] d6 



Then, 



- 1 J fl<*> ‘zjZ +1 J Pi<*> 'ZjZ2<*i-*2^’**1^*2 



z ^ z 
1 2 



z ^ z 
2 1 



Representative values of were obtained for the hyper-cube 
distance function and are listed in Table 1. Evaluation was by a 
computer program utilizing FORTRAN 63 and a sub-routine based on 
Legendre-Gaussian quadrature. Appendix I lists the program as used. 
The case Xj^» X^ and m >2 was selected for comparison reasons and 
also for reduced computer time. The program is general and allows 
selection of any parameters greater than or equal to zero. 

Time limitation prevented attainment of comparative results for 
the Euclidean distance function. 
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n 


Xf = X-2 = 20 

M-l = P-2 = 10 


= X2 = 30 
M-1 “ lJ-2 = 10 


Xj^ “ X 2 = 50 
M -1 = P <2 = 10 


1 


.3435 


.2387 


.1318 


4 


.3820 


.2842 


.1733 


8 


.3939 


.2994 


.1862 


20 


.4020 


.3071 


.1969 




o 

11 

n 


X «= X =10 


H 

II 

0 




1 2 


1 2 


1 2 


n 


= 20 


^1 “ = 30 


M-1 = IJ-2 = ^0 


1 


.5488 


.5285 


.5032 


4 


.4709 


.3989 


.3417 


8 


.4463 


.3612 


.2973 


20 


.4259 


.3327 


.2682 



TABLE 1 

Probability of Error Nonparametric Discriminator, Bivariate 
Exponential Distribution. Distance function, hyper-cube. A, 
k » 1, rule of nearest neighbor. 
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4. CONCLUSIONS AND ACKNOULEDGCMBNIS 



The application of Hodges and Fix's work to the two population 

bivariate exponential case in Section 3 resulted in representative 

results tabulated in Table 1. Though the results are small in number, 

they allow comparison with that presented by Hager in [sl* 

The nonparametric discriminator error probability values in [S] 

indicate similarity to that listed in Table 1. The case when 

X a X > jA » pi corresponds to the case c > 1 in [S]. In both [5l 
1 2 1 2 

and Table 1, probability of error increases with increase in n. 

Likewise is analgous to c < 1 and P decreases with 

12 12 1 

increasing n in both Table 1 and [f]. Table 1 does not indicate the 
sensitivity of P^ to change in parameter magnitudes nor the effect of 
the change of a single parameter. 

The intent of this thesis was to develop the explicit forms of 
the error probabilities for the two distance functions and then eval- 
uate the probabilities to determine if there is any superiority of 
one distance function over another. This goal was only partially 
attained due to the programming complexity and extensive computer time 
required to evaluate the error probability. 

The following are recommended as areas of further work: 

1. Examine sensitivity of P^ to change in parameter magnitudes 
and variation of a single parameter. 

2. Attempt to streamline program or approximation method to 
shorten computer time. 

3. Compute P^ for the Euclidean distance function to compare 
results with the hyper-cube. 
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4. Develop functional relationship of and when p = 2. 

5. Examine other distance functions. 

I wish to thank Professor J. R, Borsting for his enlightening 
guidance and assistance in preparing this thesis. I also wish to 
extend my gratitude to Mrs. Patricia Johnson for her programming 
assistance in eval.uating the equations of Section 3 and to the Computer 
Facility for their help in processing the programs. 



37 



BIBLIOGRAPHY 



1, Hodges, J. L., Jr, Discriminatory Analysis, 1. Survey of 

Discriminatory Analysis , Report prepared under contract 
between University o£ California and School of Aviation 
Medicinat Randolph Field, Texas, October 1950. 

2, Fix, Evelyn and Hodges, J, L., Jr. Nonparametric 

Discrimination, 1, Consistency Properties , Report pre- 
pared under contract between University of California and 
School of Aviation Medicine, Randolph Field, Texas. 
February 1951. 

3, Fix, Evelyn and Hodges, J. L., Jr, Nonparametric Discrim- 

ination; Small Sample Performance . Report prepared under 
contract between University of California and School of 
Medicine, Randolph Field, Texas, August 1952, 

4, Eaton, Thomas E,, Jr. Discrimination Procedures. Small 

Sample Performance , Thesis at United States Naval 
Postgraduate School, 1963, 

5, Hager, Charles F, A Comparison of Error Probabilities for 

Various Discrimination Procedures when the Populations 
are_G^naa. Thesis at United States Naval Postgraduate 
School, 1964, 



38 



b' - 



, APPENDIX I 

PROGRAMi^ii-HRECUBE 

COMMON 01 ,B2»D1»02*N,XN,IND» IMO 

1 READ 1000,N*B1»B2»D1 *D2 
PRINT2»N.B1 »B2,D1 ,02 

2 FORMAK 2X*2HN«I5*2X,3HB1 = E15.8.2X»3HB2 = E15.8.3X,3HD1*E15.8,2X.3HD2 
1=E15.8/// ) 

1000 FORMAT HSjAFIO.O) 

XN = N 
IND = 1 



CALL TRAR KAl) 

PRINT3»Ai. 

3 FORMAK 5}j,3HAl = E15.8/// ) 
IND=2 ■ 



CALL TRAP 1 (A2) 

PRINT 4»A2 

4 FORMAT! 5X,3HA2=E15. 8/// ) 

PI = XN*(A1+A2) 

PRINT 2000*N*B1*B2*D1.D2*P1 
2000 FORMAT! 2X*2HN= I 5»2X,3HB1=E 15.8 »2X 
1 3HD2 = E15.8,2X,3HP1 = E15."8/// ) 

GO TO 1 
END 

SUBROUTINE TRAPIIAREA) 

COMMON B1»B2»D1*D2»N,XN» IND.IMD 



»3H02»E15.8*2X»3KDl=ei5.8»2X» 



NN=10 
XINC = .5 
XEND = 0. 

AREA = 0. 

10 XI = XEND 

XEND = XEND +XINC 

AREAX = GLOUAD !XI»XEND.NN) 

AREA = AREA+AREAX 
IF!AREAX-l.E-06) 20.20.10 
20 RETURN 
END 

FUNCTION F!U) 

COMMON B1.B2.D1.D2.N.XN.IND.IMD 



NN=10 
XINC =.‘5 
AREA = 0. 

XEND = 0. 

IF!ABSF!U)-l.E-06) 30.30.10 
10 XI=XEND 

XEND = XEND +XINC 
IF I XEND-U) 15 .15 .12 
12 XEND = U 

15 AREAX = GLQUAD2 ! X I . XEND.NN . U ) 
AREA =AREA + AREAX 
IF!ABSF!U-XEND)-l.E-06 ) 30.30.20 
20 IF!AREAX-l.E-06) 30.30.10 
30 GO TO !40.50) .IND 
40 B=B1 



GO TO 60 



50 B=B2 

60 F « B*EXPF!-B*U)*AREA 
RETURN 
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% 



END 

FUNCTION G(V»U) 

COMMON B1»B2»D1,D2.N,XN.IND.IMD 
IMD = 1 

CALL TRAP2 (0»V»U»V.A1) 

IMD = 2 

CALL TRAP2 (V»U*U*V*A2) 

IMD=3 

CALL TRAP 3( U»U»V*A3) 

GO TO ( 400*500) ♦ IND 
400 B=B1 

GO TO 600 
500 B=B2 

600 G = B* EXPF(-B*V)*(A1+A2+A3) 

RETURN i 

END j 

SUBROUTINE TRAP2 ( A »C » U » V ♦ AREA ) ■ 

COMMON B1*B2.D1»D2*N*XN*IND*IMD 

NN=10 

XINC=.5 

AREA=0. 

XEND=A 

I F(ABSF(C)-l.E-06) 30*30.10 
10 XI=XEND 

XEND = XEND +XINC 
IF(XEND-C) 15*15*12 
12 XEND = C 

15 AREAX = GLOUAD 3 ( XI *XEND*NN*U*V) 

AREA=AREA+AREAX 

IF(ABSF(C-XEND)-l.E-06 >30*30*20 
20 IF(AREAX-1.E-06)30*30*10 
30 RETURN 
END 

SUBROUTINE TRAP3 { A *U * V * AREA ) 

COMMON B1*B2*D1*D2*N*XN*IND*IM0 

NN=10 

XINC=.5 

AREA=0. 

XEND=A 
10 XI=XEND 

XEND = XEND +XINC 

AREAX = GLOUAD 3 ( X I * X END.NN *U * V ) 

AREA=AREA+AREAX 
IF(AREAX-1,E~06)20*20*10 
20 RETURN 
END 

FUNCTION HF(X,U*V) 

COMMON B1*B2.D1*D2*N*XN*IND*IMD 
GO TO ( 100,500)*IND 
100 Z1=U 
Z2 = V 

GO TO ( 200,300*400) * IMD 

200 HF= ( 1.-4.*EXPF(-B1*Z1-B2*Z2 )* S I NH ( X , D2 ) *S I NH ( X * B1 ) ) **N* ( 1 . -4. * 
1EXPF( -D1*Z1-D2*Z2 )* S I NH ( D2 * X ) *S I NH ( D1 * X ) ) ** ( N- 1 ) * ( 4.* EXPF(-D1* 
2Z1-D2*Z2 )*(D1*SINH(D2*X)*C0SH(D1*X)+D2*SINH( Dl*X)*COSH( D2*X) ) ) 

GO TO 900 
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n o o n o 






300 HF = (1. -2.*EXPF(-B1*Z1 )* S I NH ( B1 . X ) +2 . *EXP F ( -B 1 *Z 1-B2*Z2-B2*X ) 

1* SINH( B1 tX ) )**N*( 1.- 2.*EXPF(-D1*Z1 )*SINH(D1,X) +2 . *EXPF ( -01*Z 1 
2-D2*Z2-D2*X ) * S I NH ( D1 »X ) ) ** 2 . *Dl*EXPF ( -Dl*Z 1 ) * COSH 

3 <D1»X) - 2.*D1*EXPF(-D1*Z1-D2»Z2 -D2*X) * C0SH(D1*X) +2.*D2*EXPF 

4 (-D1*Z1-D2*Z2-D2*X) *SINH(DltX)) 

GO TO 900 

400 HF= (EXPF(-B1*Z1-B1*X) +EXPF ( -B2*Z 2-B2*X ) -EXPF ( -B1*Z 1-B2*Z 2~B 1* 

1 X-B2*X)h**N* ( EXPF(-D1*Z1-D1*X) +EXPF < ~D2*Z 2~D2*X ) -EXPF ( ~D 1*Z 1- 
2D2»Z2 -QpX-D2*X) )**(N-1 ) * ( Dl*EXPF ( -Dl*Z 1-D 1»X ) +D2* EXPF(-D2» 

3 Z2-D2*X,J-( D1+D2) *EXPF ( -D1*Z 1-D2*Z2-D 1*X-D2*X ) ) 

GO TO 90'p 
500 Z1=V 
Z2 = U 

GO TO (600.700»800 ) ♦ IMD 

600 HF= (1.-4.*EXPF(-B1*Z1-B2*Z2)* S I NH ( X ♦ B2 ) *S I NH I X , B1 ) ) **N* ( 1 . -4. * 
1EXPF( -D1*Z1-D2*Z2 )* S I NH ( 02 ♦ X ) *S I NH ( Dl » X ) ) *» ( N-1 ) * ( 4 .* EXPF(-D1* 
2Z1-D2*Z2)*{D1*SINH(D2 »X)*C0SH(D1»X )+D2*SINH(Dl»X)*COSH(D2»X) ) ) 

GO TO 900 t 

700 HF=( 1. -2.*EXPF(-B2*Z2 )*SINH(B2»X)+2.*EXPF(-B1*Z1-B2*Z2-B1*X )* 

1SINH(B2*X) )**N*( 1.-2** EXPF ( -D2*Z2 ) * SINH(D2»X)+ 2.* EXPF ( -D1*Z 1-D 
22*Z2-D1*X) *SINH(D2*X) )**(N-1) * ( 2. *D2*EXPF< -02*22 ) * C0SH(02»X) 

3 -2.* 02*EXPF(-01*Z1-02*Z2-01*X)*COSH(02»X) +2. *01 *EXPF ( -01*21-02 
4*Z2 -01*X)* SINH(02*X)) 

GO TO 900 

800 HF= (EXPF(-B1*Z1-B1*X) +EXPF ( -B2*Z 2"B2*X ) -EXPF( -B1*Z1-B2*Z2-B1* 

1 X-B2*X))**N* ( EXPF( -01*Z1-01*X) +EXPF ( -02*Z 2-02*X ) -EXPF ( -01*Z 1- 
202*22 -01*X-02*X) ) ** (N-1 ) * ( Oi*EXPF ( -0l*Z1^0l#X ) +02* EXPF(-02* 

3 Z2-D2*X)-( 01+02) *EXPF (-01*2 1-02*Z2-D1#X-02»X ) ) 

900 RETURN 
ENO 

FUNCTION COSH(Y.X) 

COSH=(EXPF(Y*X)+EXPF(-Y*X) )/2, 

RETURN 

END 

FUNCTION SINH(Y.X) 

SINH=(EXPF(Y*X)-EXPF(-Y*X) )/2, 

RETURN 

END 

FUNCTION GLOUAD (A,B»N) 

01 UCSO GLOUAD F 

GAUSSIAN-LEGENDRE QUADRATURE OF F FROM A TO B» 10*20 OR 40 NODES* 

P. yager 10/20/64 (RFF. KRYLOV PP338.341 AND SEC 7.2) 



COMMON /GLODATA/Xl (5) . A 1 ( 5 ) » X2 ( 10 ) ♦ A2 ( 10 ) ♦ X4 ( 20 ) ♦ A4 ( 20 ) 

DATA (Xl= ' 

0 .9739065285* .8650633667* .6794095683* .4333953941* .1488743390) 
data (A1= 

D .0666713443* .1494513491* .2190863625* .2692667193* .2955242247) 
DATA (X2= 



D .9931285992* 
D .6360536807* 
DATA (A2= 

0 .0176140071* 
0 .1181945320* 
DATA (X4= 



.9639719273* 

.5108670020*, 

.0406014298* 

.1316886384, 



9122344283* 

3737060887, 

0626720483 , 
1420961093, 



.8391169718* 

.2277858511* 

.0832767416* 

.1491729865* 



.7463319065* 
.0765265211 ) 

.1019301198* 

.1527533871) 



41 



n rv n n o 



D .9982377097, 
D .9020988070, 
D .6719566846, 

, D .3419940908, 
\ DATA (A4= 

D .0045212771, 
D .0334601953, 
D .0574397691, 
D .0728865824, 
T0=(A+B)/2. $ 
IF(N-10n ,1,4 



.9907262387, 

.8659595032, 

.6125538897, 

.2681521850, 

.0104982845, 
.0387821680, 
.0613062425, 
.0747231691, 
T1=(B-A) /2. 



.9772599500, 
.8246122308 , 
.5494671251, 
.1926975807, 

.0164210584, 
,0438709082, 
.0648040135, 
.0761103619 , 
$ Y = 0 



.9579168192 , 
.7783056514, 
.4830758017, 
.1160840707, 

.0222458492* 
,0486958076, 
,0679120458, 
,0770398182 , 



.9328128083, I 
,7273182552, 
.4137792044, 
.0387724175) 



.0279370070, 
,0532278470, 'j 
.0706116474, • 

.0775059480) 



1 DO 2 K=l,5 

2 Y=Y+A1(K)*(F(T0-T1*X1(K) ) +F ( T0+T1#X1 ( K ) ) ) 

3 GLQUAD=Y*T1 1 

REjtURN I 

4 IF(N-20)5,5,7 ! 

5 DO 6 K=l,10 

6 Y=Y+A2(K)*(F(T0-T1*X2(K) )+F(T0+Tl*X2(K) ) ) ; 

GOTO 3 I 

7 DO 8 K=l,20 

8, Y=Y+A4(K)*(F(T0-T1*X4(K) )+F(T0+T1*X4(K) ) ) 

GOTO 3 
END 

FUNCTION GLOUAD2 (A,B,N,U) 

D1 UCSD GLOUAD 

GAUSSIAN-LEGENDRE QUADRATURE OF F FROM A TO B, 10,20 OR 40 NODES. 



P. YAGER 10/20/64 (REF. KRYLOV PP338,341 AND SEC 7,2) 






COMMON /GLQDATA/Xl ( 5 ) , A 1 ( 5 ) , X2 ( 10 ) , A2 ( 1 0 ) , X4 ( 20 ) , A4 ( 20 ) 

DATA (Xl= 

D ,9739065285, .8650633667, ,6794095603, .4333953941, ,1488743390) ' 



data (A1= 

D .0666713443, 
DATA (X2= 

D .9931285992, 
D .6360536807, 
DATA (A2= 

D .0176140071, 
D .1181945320, 
data (X4= 

D .9982377097, 
D .9020988070, 
D .6719566846, 
D .3419940908, 
data (A4= 

D .0045212771, 
D .0334601953, 
D .0574397691, 
D .0728865824, 
T0=(A+B)/2. $ 
IF(N-10)1 ,1,4 

1 DO 2 K=l,5 

2 Y=Y+A1( K)*(G( 

3 GL0UAD2 *Y*T1 
RETURN 



.1494513491, .2190863625, .2692667193, 



.9639719273, .9122344283 
.5108670020, .3737060887 

.0406014298, .0626720483 
.1316886384, .1420961093 



.9907262387, 

.8659595032, 

.6125538897, 

.2681521850, 

.0104982845, 
.0387821680, 
.0613062425, 
.0747231691 , 
Tl=(B-A)/2. $ 



.9772599500 

.8246122308 

.5494671251 

.1926975807 

.0164210584 
.0438709082 
.0648040135 
.0761103619 
Y = 0 



, .8391169718, 
, .2277858511, 

, .0832767416, 
, .1491729865, 

, .9579168192, 
, .7783056514, 
, .4830758017, 
, .1160840707, 

, .0222458492, 
, .0486958076, 
, .0679120458* 
, .0770398182* 



T0“T1*X1(K) *U)+G(T0+T1*X1(K) ,U) ) 



.2955242247) 

.7463319065* 
.0765265211 ) 

.1019301198, 

.1527533871) 

.9328128083, 

.7273182552, 

.4137792044, 

.0387724175) 

.0279370070, 
.0532278470, . 
.0706116474, , 
.0775059480) 



AO 



n n n n n 



f 



4 IF(N-20^5»5,7 

5 DO 6 K=l»10 

6 Y=Y+A2(K)*(G(T0-T1*X2(K) » U ) +G ( TO+T 1 *X2 ( K ) »U) ) 

GOTO 3 ■ 

7 DO 8 K=i*20 

8 Y=Y+A4(I<)*(G(T0-T1*X4(K)»U)+G(T0+T1*X4(K) ,U) ) 

GOTO 3 

END 

FUNCT ION .GLOUAD3 (A,B»N,U.V) 

D1 UCSD GLQUAD F 

GAUSSIAN-LEGENDRE QUADRATURE OF F FROM A TO B» 10*20 OR 40 NODES. 



P. yager 10/20/64 (REF. KRYLOV PP338»341 AND SEC 7.2) 



D 



COMMON /GLQDATA/Xl (5)»A1(5).X2(10).A2(10)»X4(20 
DATA (Xl= 

.9739065285. .8650633667. .6794095683. .433395 



) .A4(20) 

3941. .1488743390) 



1 

2 

3 

4 

5 

6 

7 

8 



DATA (Al= 

D .0666713443. 

DATA (X2= 

D .9931285992. 
D .6360536807. 

DATA (A2= 

D .0176140071. 
D .1181945320. 

DATA (X4= 

D .9982377097. 
D .9020988070. 
D .6719566846. 
D .3419940908. 

DATA (A4= 

D .0045212771. 
D .0334601953. 
D .0574397691. 
D .0728865824. 
T0=(A+B)/2. $ 
IF(N-10)1.1.4 
DO 2 K=1.5 



.1494513491. 

.9639719273. 

.5108670020. 

.0406014298. 

.1316886384. 

.9907262387. 

.8659595032. 

.6125538897. 

.2681521850. 

.0104982845. 
.0387821680. 
.0613062425. 
.0747231691. 
T1=(B-A) /2. 



.2190863625. 

.9122344283. 

.3737060887. 

.0626720483. 

.1420961093. 

.9772599500. 
.8246122308. 
.5494671251 . 
.1926975807. 

.0164210584. 
.0438709082. 
.0648040135. 
.0761103619. 
$ Y=0 



.2692667193. 



.8391169718. 
.2277858511 . 



.0832767416. 

.1491729865. 

.9579168192. 

.7783056514. 

.4830758017. 

.1160840707. 



.0222458492 . 
.0486958076. 
.0679120458. 
.0770398182. 



Y=Y+A1(K)*(HF(T0-T1*X1 (K) .U . V ) +HF ( TO+T 1*X 1 ( K ) .U»V) ) 

GLQUAD3 =Y*T1 

RETURN 

IF(N-20)5.5.7 
DO 6 K=1.10 

Y=Y+A2(K)*( HF (T0-T1*X2(K) .U. V ) +HF ( TO+T 1*X2 ( K ) .U»V) ) 
GOTO 3 
DO 8 K=1.20 

Y=Y+A4(K)*(HF(T0-T1*X4(K) .U . V ) +HF ( TO+T 1 *X4 ( K ) .U* V ) ) 

GOTO 3 

END 



.2955242247) 

.7463319065. 
.0765265211 ) 

.1019301198. 
.1527533871 ) 

.9328128083. 

.7273182552. 

.4137792044. 

.0387724175) 

.0279370070. 

.0532278470. 

.0706116474. 

.0775059480) 



END 

FINIS 



-EXECUTE 
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